AD-A225  540 


*  t 


OTIC 


ELECTE 
AUG  1  G  1990 


/N 

6 


L / 


CCS  Research  Report  No.  651 

Suggested  Research  Topics  in  Sensitivity 
and  Stability  Analysis  for 
Semi-Infinite  Programming  Problems 

by 

A.  Fiacco* 

Y.  Ishizuka* 


April  1990 


*George  Washington  University  School  of  Engineering  and  Applied  Science,  Department 
of  Operations  Research,  Washington,  D.C.,  20052 

This  research  was  partly  supported  by  National  Science  Foundation  Grant  SES-8722504 
with  the  Center  for  Cybernetic  Studies,  The  University  of  Texas  at  Austin,  and  National 
Science  Foundation  Grant  ECS-86- 19859  and  Office  of  Naval  Research  Grant  N000 14-89- 
J- 1537  with  the  School  of  Engineering  and  Applied  Science,  George  Washington 
University.  Reproduction  in  whole  or  in  part  is  permitted  for  any  purpose  of  the  United 
States  Government 


l 


'  5TATSy.SNT  A 

Appiov«Q  to:  puriiC  teleoaot 
Diitncuuoc  Utilinuiod _ 


CENTER  FOR  CYBERNETIC  STUDIES 
A.  Chames,  Director 


College  of  Business  Administration,  5.202 
The  University  of  Texas  at  Austin 
Austin,  Texas  78712-1177 
(512)  471-1821 


02  I  S'  IS1-/ 


SUGGESTED  RESEARCH  TOPICS  IN  SENSITIVITY 
AND  STABILITY  ANALYSIS  FOR 
SEMI-INFINITE  PROGRAMMING  PROBLEMS 


Abstract 

i 

s 

We  suggest  several  important  research  topics  for  semi-infinite  programs  whose  problem 
functions  and  index  sets  contain  parameters  that  are  subject  to  perturbation.  These  include 
optimal  value  and  optimal  solution  sensitivity  and  stability  properties  and  penalty  function 
approximation  techniques.  The  approaches  proposed  are  a  natural  carryover  from 
parametric  nonlinear  programming,  with  emphasis  on  practical  applicability  and 
computability. 
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Semi-infinite  programming,  perturbation  analysis,  sensitivity  analysis,  stability,  parametric 
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SUGGESTED  RESEARCH  TOPICS  IN  SENSITIVITY 
AND  STABILITY  ANALYSIS  FOR 
SEMI-INFINITE  PROGRAMMING  PROBLEMS 


by 

Anthony  V.  Fiacco 
and 

Yo  Ishizuka 


1.  INTRODUCTION 

Our  purpose  is  to  suggest  potentially  fruitful  areas  of  research  in  sensitivity 
and  stability  analysis  for  semi-infinite  programming  problems  whose  objective  and 
constraint  functions  and  index  sets  are  perturbed  by  several  parameters.  The  main 
subjects  of  this  paper  are  the  following: 

I.  Basic  sensitivity  and  stability  results:  optimal  value  and  optimal  solution 
sensitivity  and  stability  properties,  i.e.  continuity,  differentiability,  and 
lower  and  upper  bounds  of  optimal  values  and  optimal  solutions  of 
parametric  semi-infinite  programming  problems; 

II.  Development  of  the  penalty  function  method  as  a  solution  strategy  and 
also  as  an  algorithmic  approximation  technique  for  sensitivity  information 
in  semi-infinite  programming; 

III.  Computational  implementation. 


The  proposed  areas  of  research  are  envisioned  as  extensions  of  the  respective 
sensitivity  and  stability  results  and  techniques  for  the  standard  nonlinear 
programming  problem  (NLP): 

A.  Basic  sensitivity  and  stability  characterizations,  parameter 
derivatives  and  bounds. 

B.  Penalty  function  methodology. 

C.  Computer  codes  for  solution,  sensitivity  and  bound  analysis  for 
NLP,  and  demonstrations  of  practical  applicability. 

Throughout  this  paper,  emphasis  will  be  placed  on  computable  results  and 
computational  aspects. 

2.  SEMI-INFINITE  PROGRAMMING 
2.1  Formulations  and  Applications 

The  semi-infinite  programming  problem  (SIP)  is  a  nonlinear  programming 
problem  formulated  as  follows: 

SIP:  min  F(x) 

x 

s.t.  Gj(x,  tj)  >  0  Vtj  e  Tj,  i=l,  ...  ,  m, 

where  F:  En  -  E1,  Gj:  En  x  E**1  -  E1  ,  Tj  C  EP‘  ,  and  Eq  is  Euclidean  q-space. 
The  sets  Ti  are  called  index  sets.  We  assume  at  least  one  Tj  consists  of  infinitely 
many  elements.  The  linear  semi-infinite  programming  problem  is  a  special  case  of 
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SIP  and  is  formulated  as: 


LSIP:  min  c^x 

x 

s.t.  x  >  bj(tj)  Vtj  €  Tj,  i— I,  ...  ,  m, 

where  c,  aj(tj)  £  En,  bj(tj)  £  e\  and  the  superscript  “T”  denotes  “transpose.” 

SIP  arises  in  various  kinds  of  system  optimizations.  Roughly  speakir*®, 
optimization  problems  that  are  naturally  formulated  as  SIP  may  be  classified  into 
three  categories,  according  to  the  physical  meanings  of  the  index  sets  Tj. 

(i)  Decision  problems  under  uncertainty:  the  case  where  tj  e  Tj  stands  for 
the  i-th  “unknown”  (i.e.,  the  i-th  opponent  or  i-th  disturbance)  variable, 
and  the  decision— maker  wishes  to  optimize  his  objective  F  under  the 
condition  that  the  performance  indexes  Gj  be  kept  over  the  permissible 
levels  (zero),  whatever  the  opponent’s  decision  may  be  or  whatever 
disturbances  may  happen.  Examples:  The  problem  of  designing  frame 
buildings  (Polak  (1983)],  taking  into  account  seismic  force  as  a 
disturbance,  and  electronic  circuit  design  problems  [Polak  (1983)]  that 
include  a  production  error  parameter  as  the  unknown  factor.  (These 
formulations  by  Polak  also  include  time-dependent  constraints,  mentioned 
below.); 

(ii)  The  case  when  the  constraints  depend  on  “time”  or  “space,”  i.e., 

tj  e  Tj  stands  for  time  or  space  or  frequency  variables.  Examples:  An 
air  pollution  control  problem  [see  Glashoff  and  Gustafson  (1983)]  which 
requires  the  minimum  cost  strategy  to  keep  concentrations  of  a  pollutant 
below  the  permissible  level  at  every  point  in  an  area,  and  the  design  of 
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feedback  control  systems  (Polak  (1983)J,  over  time  or  frequency  domains; 
(iii)  Min-max  (or  Chebyshev  or  Uniform)  approximation  problems:  Given  a 
function  f  and  a  family  A  a  {a(x,  -)  I  x  e  X)  of  approximation  functions, 
the  min-max  approximation  problem  (MAP)  on  a  domain  T  is  formulated  as 
follows: 

MAP:  min  max  |  a(x,  t)  —  f(t)  |  . 
xeX  teT 

This  problem  can  be  rewritten  as  an  SIP: 

MAP:  min  a 

(x,c r) 

s.t.  a(x,  t)  —  f(t)  +  a  >  0  Vt  e  T 
-  a(x,  t)  -(-  f(t)  +  o’  >  0  Vt  e  T  . 

In  particular,  when  a (•  ,  t)  is  linear  in  x,  MAP  is  a  special  case  of  LSIP.  Most  of 
the  work  for  SIP  and  LSIP  is  motivated  by  MAP  or  its  extensions. 

2.2  Existing  Results  for  SIP  and  LSIP 

As  for  nonlinear  programming  (NLP),  many  aspects  of  SIP  or  LSIP  have  been 
studied,  e.g., 

•  Optimality  conditions: 

•  First  order  conditions  [John  (1948),  Gehner  (1974),  Hettich  and  Jongen 
(1978)1; 

•  Second  order  conditions  [Ben-Tal,  Teboulle  and  Zowe  (1979),  Ioffe  (1983), 
Shapiro  (1985)]; 
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•  Duality  theory  [see  papers  in  Fiacco  and  Kortanek  (1983),  Glashoff  and 
Gustafson  (1983),  Anderson  and  Nash  (1987)]; 

•  Solution  methods  for  LSIP: 

•  Exchange  algorithms,  dual  simplex  methods  and  cutting  plane  methods 
[Hettich  (1983),  Glashoff  and  Gustafson  (1983)]; 

•  Primal  simplex  methods  [Anderson  (1985),  Anderson  and  Nash  (1987)]; 

•  Solution  methods  for  SIP: 

•  Cutting  plane  method  [Blankenship  and  Falk  (1976)]; 

•  Lagrange  methods  [Watson  (1983,  1985),  Coope  and  Watson  (1985)]; 

•  Exact  penalty  method  [Conn  and  Gould  (1987)]. 

The  difficulty  of  dealing  with  SIP  or  LSIP,  of  course,  is  the  presence  of  an 
infinite  numbers  of  constraints.  A  useful  technique  to  overcome  this  difficulty 
replaces  the  infinitely  many  constraints  with  a  finite  number,  without  changing  the 
optimal  value  [Borwein  (1981,  1983),  Jongen,  Jonker  and  Twilt  (1983)].  It  is  well 
known  that  this  reduction  is  possible  under  mild  assumptions  (e.g.,  convexity).  This 
fact  forms  a  basis  for  a  duality  theory,  and  hence,  for  dual  simplex  methods. 
Furthermore,  under  stronger  assumptions  [e.g.,  assumptions  under  which  the  basic 
sensitivity  theorem  by  Fiacco  (1983,  1976)  holds]  for  the  inner  problem, 

IPj(  x  ):  min  G{(  3f,  tj  , 

1  teT-  1  1 

lr  1  i 

of  SIP  at  If,  we  can  obtain  a  finite  number  of  functions  t-  ,  j  =  l,  ...,  o-j,  i=l,...,  m 
which  (locally)  solve  IPj(x)  for  x  e  NOf)  (a  neighborhood  of  If),  and  then  it  can  be 
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shown  that  SIP  is  (locally)  equivalent  to  the  following  problem: 


RSIP:  min  F(x) 

x 

s.t.  Gjfx,  tj j(x>]  >  0,  j  =  l,...,ai,  i  =  l,  ....  rn. 

If  t|j  e  C1,  the  reduced  problem  RSIP  is  a  standard  NLP,  and  the  theory  for  NLP  will 
be  applicable  to  RSIP.  This  idea  is  widely  used  in  theoretical  developments  [Shapiro 
(1985),  Zwier  (1987)]  and  numerical  solution  methods  [Hettich  (1983),  Watson 
(1983,  1985),  Conn  and  Gould  (1987)].  An  extension  of  this  technique  should  be 
useful  in  parametric  SIP  as  well  (see  Section  3.2). 

3.  PROPOSED  RESEARCH  TOPICS 
3.1  Parametric  SIP 

We  are  concerned  with  the  question:  If  the  data  involved  in  an  SIP  changes, 
what  happens  to  the  optimal  value  or  the  optimal  solutions?  For  example,  let  us 
consider  the  application  problems  mentioned  in  Section  2.1. 

(i)  Decision  problems  under  uncertainty.  If  the  decision-maker  changes  the 
permissible  levels  of  the  performance  indices  G*,  or  if  he  assumes  slightly 
different  constraint  sets  Tj  of  the  opponent  or  disturbance  variables,  how 
will  the  optimal  decision  change?  Note  that  it  is  important  to  consider 
the  latter  case  because,  in  most  situations,  the  T^  are  only  “estimated”  or 
“assumed”  sets  (e.g.,  the  set  of  expected  seismic  forces). 

(ii)  Constraints  depending  on  time  or  space  coordination.  How  do  changes  in 
cost  function,  permissible  levels,  and  area  of  air  pollution  control  affect 
the  optimal  strategy?  If  we  consider  a  different  time  interval  on  a 
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different  frequency  domain,  what  will  happen  to  the  control  system? 
(iii)  Min-max  approximation.  If  we  change  the  function  f  and  the  domain  T, 


slightly,  what  can  we  expect? 

In  order  to  answer  these  questions,  we  have  to  consider  the  parametric  semi-infinite 
programming  problem.  Introducing  the  parameter  vector  e  €  E**  in  SIP  explicitly,  we 
formulate  this  problem  as  follows: 

SIP(e):  min  F(x,  e) 
x 

s.t.  Gj(x,  e,  tp  >  0  Vtj  c  Tj(e),  i  =  l,  ...,  m  . 

We  define  the  constraint  set,  the  optimal  value,  and  the  optimal  solution  set  as 
follows: 

R(e)  =s  {x  €  En|  Gj(x,  e,  tj)  >  0  Vti  e  Tj(e),  i  =1,  m), 

F*(e)  =  inf  F(x,  e), 
xcR(e) 

S <e)  =  (x*e  R(e)|  Fix*  ,  e)  =  F*(e))  . 

As  briefly  mentioned  in  Section  1,  our  proposed  research  includes: 

I.  Basic  sensitivity  and  stability  results  for  F*(e)  and  S(e); 

II.  Penalty  function  approaches  for  SIP(e); 

III.  Emphasis  on  computability  and  computational  implementation. 

In  the  following  three  sections  we  shall  discuss  these  subjects  in  more  detail. 
First,  we  mention  important  relevant  work  in  parametric  semi-infinite  programming. 
For  standard  (i,e.,  smooth  and  finitely  constrained)  parametric  NLP,  many  useful 
results  have  been  obtained.  A  survey  of  basic  theoretical  results,  and  the 
computable  algorithmic  approach  to  parametric  NLP  is  given  in  the  book  by  Fiacco 
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(1983).  As  for  parametric  SIP,  however,  there  have  not  been  as  many  results.  In 
his  book,  Brosowski  (1982)  studies  several  types  of  perturbed  SIP  with  fixed  (i.e., 
nonperturbed)  index  sets,  and  investigates  their  “qualitative”  properties  (e.g., 
continuity  of  F*  and  S*,  etc.).  Niirnberger  (1985)  studies  the  (strong)  unicity  (i.e., 
uniqueness)  of  S(e)  of  problems  similar  to  those  studied  by  Brosowski.  If,  as  in 
Brosowski’s  formulations,  the  index  sets  T-  are  not  perturbed,  i.e.,  if  the  constraints 
in  SIP(e)  are  of  the  following  form: 


Gj(x,  e,  tp  >0  Vtj  e  Tj  ,  i=l,  ....  ra  , 

then  one  can  apply  sensitivity  and  stability  results  of  parametric 

infinite  programming  (Penot  (1984),  Lempio  and  Maurer  (1980),  da  Silva  (1985)],  which 

treat  perturbed  constraints  such  as 


Gj(x,  e,  •  )  €  Cj  ,  i*=l,  ...,  m  , 

where  Cj  is  a  positive  cone  in  a  space  of  functions  defined  on  Tj. 

Another  approach  may  be  possible:  defining  minimal  value  functions  of  the 
inner  problems  of  SIP(e)  as 

Gf(x,  e)  as  inf  Gj(x,  e,  t;),  i=l,  ...  ,  m  , 

1  tjeTjto)  1  1 

we  can  rewrite  SIP(e)  as: 

SIP*(e):  min  F(x,  e) 

x 

s.t.  G*(x,  e)  >  0,  i=l,  ...,  m  , 
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which  is  of  a  standard  parametric  NLP  form,  with  non-smooth  constraint  functions 
G*.  One  can  then  apply  sensitivity  results  from  non— smooth  parametric  NLP  (e.g., 
Auslender  (1979),  Rockafellar  (1982)]  to  SIP*(e)  to  obtain  estimates  of  the  directional 
derivatives  of  F*  in  terms  of  the  generalized  gradients  (Clarke  (1975)]  of  G*. 
Recently,  Zencke  and  Hettich  (1987)  obtained  the  directional  derivative  of  F*  of  a 
parametric  linear  SIP  with  fixed  index  sets,  via  duality  arguments. 

For  min-max  approximation  problems,  stability  properties  including  the 
perturbation  of  the  respective  domains  have  been  studied,  e.g.,  Flachs  (1985)  and 
Dunham  (1977)  investigated  the  continuity  of  the  best  rational  min-max 
approximation  with  respect  to  the  domains. 

Compared  with  past  studies,  our  proposed  research  will  have  new  and 
significant  extensions  in  the  following  respects: 

•  Allowing  the  index  sets  to  be  perturbed; 

•  Emphasizing  computability  and  computational  aspects. 

3.2  Basic  Sensitivity  and  Stability  Analysis  for  SIP(e) 

Basic  sensitivity  and  stability  analysis  for  NLP  includes  the  following 
important  areas: 

a.  Local  qualitative  analysis,  i.e.,  continuity  of  optimal  values  or  optimal 
solutions  set  (see  Section  2.2  in  Fiacco  (1983)]; 

b.  Local  quantitative  analysis,  i.e.,  estimation  and  characterization  of 
(directional)  derivatives  of  optimal  values  and  optimal  solutions  [see 
Sections  2.3  and  2.4  in  Fiacco  (1983)]; 

c.  Global  qualitative  analysis,  i.e.,  convexity  and  concavity  of  optimal  values 
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with  respect  to  the  parameter  (see  Fiacco  and  Kyparisis  (1986)]. 
d.  Global  quantitative  analysis,  i.e.,  estimation  of  lower  and  upper  bounds  of 
optimal  values  and  optimal  solutions  (see  Chapter  9  in  Fiacco  (1983), 

Fiacco  and  Kyparisis  (1988)] 

We  suggest  further  investigation  of  the  analogous  properties  and  measures 
for  SIP(e).  As  mentioned  in  the  previous  section,  viewing  SIP(e)  as  a  non-smooth 
parametric  NLP,  we  can  use  general  sensitivity  theory  to  obtain  an  estimate  of  the 
directional  derivative  of  F*.  Another  theoretically  and  computationally  useful 
approach  will  be  the  extension  of  the  reduction  idea  mentioned  in  Section  2.2  to  our 
SlP(e).  Suppose  that  the  inner  problem: 


IPjCx,  e):  min  G{ (x,  e,  t;) 

1  VTVfe)  1  1 


at  (x,  e")  satisfies  the  standard  assumptions  in  Fiacco’s  basic  sensitivity  theorem 
(Fiacco  (1983)],  at  every  minimal  solution  t*.  Then  there  exists  a  finite  number  of 
perturbed  local  minimal  solutions  ty  (x,  e),  j  =  l,  ....  of  IPjXx,  e)  for 
(x,  e)  €  N(x,  e"),  and  SIP(e)  is  (locally)  equivalent  to  the  following  reduced  problem: 

RSIP(e):  min  F(x,  e) 

x 

s.t.  Gjtx,  e,  ty(x,  e)I  >  0,  j=l,  ...,  crj,  i=l,  ...  ,  m  . 

S.nce,  under  the  assumptions,  ty  are  in  C*,  we  can  apply  existing  smooth  parametric 
NLP  sensitivity  and  stability  results  to  RSIP(e)  to  describe  the  local  behavior  of 
optimal  values  and  optimal  solutions  of  SIP(e). 
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As  for  global  properties  (i.e.,  c  and  d  above]  of  SIP(e),  (especially  for  optimal 


values),  Fiacco’s  calculation  technique  for  bounds  on  optimal  values  (Chap.  9  in 
Fiacco  (1983)]  will  be  applicable  in  a  direct  manner,  since  it  is  based  on  the 
convexity  properties  of  the  problem  rather  than  on  the  finiteness  or 
differentiability  of  the  constraints. 

3.3  Penalty  Function  Methods  for  SIP(e) 

For  standard  parametric  NLP,  Fiacco  (1983)  has  established  a  technique  to 
estimate  sensitivity  information  (e.g.  a  perturbed  Karush-Kuhn-Tucker  triple  and  its 
derivative)  via  penalty  function  algorithms  (see  also  Armacost  and  Fiacco  (1977, 
1978)].  By  this  technique,  one  can  obtain  sensitivity  information  from  trial  points 
generated  by  a  penalty  function  algorithm  for  solving  a  given  NLP.  Since  it  is  easy 
to  implement  this  technique  in  any  penalty  function  algorithm,  it  provides  a  very 
useful  sensitivity  analysis  tool  in  practice. 

We  suggest  the  adaptation  of  this  technique  to  SIP(e).  To  this  end,  of 
course,  a  penalty  function  algorithm  for  solving  SIP  must  be  established. 
Unfortunately,  however,  it  seems  that  no  serious  studies  of  penalty  methods  for  SIP 
(or  LSIP)  have  been  made,  except  for  Conn  and  Gould’s  recent  exact  penalty 
function  method  for  SIP  (Conn  and  Gould  (1987)].  We  propose  to  investigate 
application  of  classical  penalty  (especially,  barrier)  function  methods  to  solve  SIP 
problems.  For  example,  denoting  the  penalty  parameter  by  r, 


P(x,  r) 

■  F(x)  + 

Ul 

z 

i-1 

Mi 

G/x,  V  dti  ’ 

P(x,  r) 

m 

m 

■  F(x)  - 

z 

i-1 

JtieTi 

r  £n(Gj(x,  tp]  dtj, 
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are  simple  extensions  of  classical  barrier  functions.  It  seems  clear  that  a  slight 
modification  of  finite-dimensional  standard  proofs  of  convergence  will  yield 
convergence  theorems  of  methods  for  SIP  based  on  these  barrier  function  methods. 
Since  the  publication  of  Karmarkar  (1984),  many  serious  attempts  to  reconsider 
penalty  function  methods  as  solution  methods  for  LP  have  been  pursued  (e.g..  Gill, 
Murray,  Saunders,  Tomlin  and  Wright  (1986);  Megiddo  (1986);  Reneger  (1988);  Ben 
Daya  and  Shetty  (1988)1,  and  it  has  been  reported  that,  at  least  for  some  particular 
classes  of  LPs,  these  “new”  interior  path  methods  are  superior  to  the  simplex 
method.  In  addition  to  the  need  to  develop  practical  sensitivity  analysis  tools,  this 
is  another  motivation  for  investigating  the  penalty  function  approach  for  solving 
SIP.  Realizing  that  many  existing  algorithms  for  SIP  (especially  LSIP)  can  be 
regarded  as  (dual  or  primal)  simplex  methods  [Hettich  (1983),  Glashoff  and  Gustafson 
(1983),  Anderson  and  Nash  (1987)],  and  that  the  effort  to  construct  solution  methods 
(even  for  LSIP)  continues,  we  are  confident  that  a  penalty  function  approach  will 
provide  an  efficient  solution  method  as  well  as  an  important  sensitivity  analysis 
tool  for  SIP. 

3.4  Computational  Implementation 

Based  on  the  penalty  function  approach  mentioned  in  the  previous  section, 
Armacost  and  Fiacco  (1978)  developed  a  computer  code  called  SENSUMT  to  estimate 
the  sensitivity  information  of  a  standard  parametric  NLP.  As  a  preliminary 
experiment,  we  propose  the  use  of  SENSUMT  to  analyze  SIP(e).  Having  solved  the 
inner  problem  IP^x,  e)  by  an  appropriate  NLP  algorithm,  one  may  apply  SENSUMT  to 
the  reduced  problem  RSIP(e)  to  obtain  sensitivity  information  for  SIP(e). 
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We  also  suggest  investigation  of  the  use  of  integral  types  of  barrier 
functions  like  those  mentioned  in  the  previous  section.  Since  we  do  not  have 
experimental  results  for  such  barrier  methods,  their  efficacy  remains  to  be  tested 
computationally. 

3.5  Disclaimer 

We  do  not  presume  to  include  ail  the  vital  issues  and  important  current 
references  in  SIP,  e.g.,  incisive  work  in  the  stability  of  the  feasible  set  recently 
studied  by  Jonger,  Twilt  and  Weber  (“Semi-Infinite  Optimization:  Structure  and 
Stability  of  the  Feasible  Set,”  Memo.  No.  838,  University  of  Twente,  Enschede,  The 
Netherlands,  December  1989].  (See  also,  references  to  recent  work  in  this  paper.) 
The  topics  addressed  are  simply  those  of  particular  interest  to  the  authors  and 
deemed  to  be  of  considerable  importance  among  those  for  which  computable  results 
may  be  obtained.  Finally,  we  note  that  all  the  results  addressed  in  this  paper  can 
be  extended  to  include  equality  constraints.  Since  this  extension  is  straightforward, 
we  do  not  pursue  the  details  here. 
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